For the purposes of this specification, the present invention will generally be described in relation to controlling drones by predicting the positions of objects based on delays in human perception and response time. However, the invention is not so limited and may be applied in a variety of other applications where effectively eliminating the delays in perception and response time may be beneficial including, but not limited to, online gaming, remote surgery, remote robotic devices, control or response to elements moving faster than anticipated by biologically calibrated flash-drag compensation, and other remote and/or rapidly moving situations and/or targets.
Human sensory processing is not instantaneous. There are significant delays between generation of data (e.g., photons bouncing off of an object), reception of the data (e.g., photons reaching the retina), transmission of the data via nerves, and processing of the data by the brain.
Looking at a specific example, the time between photons bouncing off of an object in Iraq, being received in a drone camera, being processed by the drone camera and computer system, and encryption of the data, represents hundreds or thousands of milliseconds of delay. Transmission to a geostationary satellite and retransmission from the satellite to a Nevada ground station takes approximately 250 milliseconds. Decryption, processing and presentation to a drone pilot at that ground station takes additional milliseconds. It would not be unusual for the delay between generation of data in Iraq and action by a Nevada-located pilot based on that data to add more than one second to the time between data generation and implementation of the instructions sent to the drone in response.
Looking at a biological example, delays in human data processing mean that a bird in flight is slightly forward of the position it was in at the time of generation of the data that underlies the human perception of the bird (the time at which the visual data was generated by light reflecting off of the bird). Evolution provides humans with a brain function known as the “flash-drag effect,” whereby the mammalian brain alters the apparent position of perceived objects to match the position they will be in at the time the brain is presented with the data received in the retina. As a result, for example, a hunter will “see” a flying bird at a point forward of where the photons hitting the retina actually reveal the bird to be located. Without the flash-drag effect, human hunters would constantly be shooting at a prior position of a moving target.
The flash-drag effect is calibrated for the circumstances for which it evolved—namely, unaided observation of objects in the direct field of view. As a result, it is unable to compensate for the additional delays caused by processing of data generated at a distance and/or in a non-biological manner. Similarly, the compensation mechanisms evolved at a time when targets and threats moved at a far slower speed and in a different manner than modern targets and threats. For example, a speeding car travels faster than the 65 miles per hour of the fastest land animal, the cheetah. The benefits of the flash-drag effect were significant enough that the effect evolved and persisted. However, it is less efficacious in modern world, and there is a strong need for a technology that allows similar benefits to be enjoyed with regard to modern threats, modern targets, non-biologically generated data, and actions over a large distance.
As AR technology becomes more widely adopted as a mechanism for control of or interaction with remote devices (and in some applications, even with respect to control of local devices), compensation for delays in transmission and processing of data, together with mechanisms to address instances where the compensation turns out to be inaccurate or insufficient, will be of enormous value in improving the range and utility of AR interfaces.
Consequently, there is a strong need for systems, methods and devices that effectively control remote devices by compensating for the delay in human perception and response time. To this end, it should be noted that the above-described deficiencies are merely intended to provide an overview of some of the problems of conventional systems, and are not intended to be exhaustive. Other problems with the current state of the art and corresponding benefits of some of the various non-limiting embodiments may become further apparent upon review of the following description of the invention.